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REMOTE CONSOLE FOR 
NETWORK APPLICATION SERVERS 

This application is a continuation in part of Serial No. 09/ 465^982, filed 
on December 16, 1999. 

5 FIELD OF THE INVE^JTION 

The present invention relates generally to field of image retrieval and 
organization. More specifically, the present invention is directed to retrieving 
and organizing multimedia objects stored on networked services. 

BACKGROUND 

10 Multimedia is a term used to describe the combined use of different 

kinds of information. Examples of the different kinds of information include 
text documents, music, images, voice recordings, etc. The combined 
information is sometimes referred to as multimedia objects. 

As the World Wide Web (WWW) becomes more popular, it is apparent 

15 that multimedia plays a critical role in effectively exchanging and 

disseminating information among users of the web. Multimedia can be used in 
many different applications, such as, for example, digital story telling, 
employee training, online music video, document with voice annotations, etc. 
As the use of multimedia expands, there is a need for storing and 

20 retrieving the multimedia objects. In order for the multimedia objects to be 
easily retrievable, the multimedia objects have to be stored in a location where 
everyone can share and access them. One way of doing this is to store the 
multimedia objects on servers connected to the Internet. Using the HTTP 
protocol, a user from anywhere in the world can access the multimedia objects 

25 as long as that user knows about the URL associated with the multimedia 
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objects. However, the user may not have cor\trol over the server(s) in order to 
organize the media objects according to their retrieval needs. 

One main requirement to this approach is that in order to access the 
Internet, the user needs to have an Internet access device such as, for example, a 
5 personal computer. What is needed is a way to organize the multimedia objects 
such that users can easily access them. 
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SUMMARY OF THE INVENTION 



A method and system that organize and retrieve multimedia objects are 
described. In one embodiment, a controller is configured to select an identifier 
5 associated with a media object to send a request to play the media object. The 
controller sends the request by transmitting the identifier stored in the 
controller. An appliance is configured to receive the request from the 
controller. The appliance retrieves the media object from a first server via a 
network connection when the media object is not stored in the appliance. The 
10 appliance plays the media object. 
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BRIEF DESCRIPTION OF THE DRAWINGS 
The present invention is illustrated by way of example in the following 
drawings in which like references indicate similar elements. The following 
drawings disclose various embodiments of the present invention for purposes 
5 of illustration only and are not intended to limit the scope of the invention. 

Figure 1 illustrates an exemplary network embodiment with a remote 
controller, a network access appliance and a server. 

Figure 2 is an exemplary diagram illustrating the remote controller 
accessing multiple network access appliances. 
10 Figure 3 is an exemplary flow diagram illustrating a request to play a 

media object. 

Figure 4 is an exemplary flow diagram illustrating an access 
authorization. 

Figure 5 is an exemplary flow diagram illustrating a request to acquire a 
15 media object. 

Figure 6 illustrates an exemplary embodiment of a remote controller. 

Figure 7 illustrates an exemplary representation of a remote controller 
with a stereo network access appliance. 

Figure 8 illustrates an exemplary representation of a remote controller 
20 with a document network access appliance. 
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DETAILED DESCRIPTION 



A remote control system for network access appliances is disclosed. The 
following detailed description sets forth numerous specific details to provide a 
thorough understanding of the invention. However, those of ordinary skill in 

5 the art will appreciate that the invention may be practiced without these 

specific details. In other instances, well-known methods, procedures, protocols, 
components, algorithms, and circuits have not been described in detail so as not 
to obscure the invention. 

Some portions of the detailed descriptions that follow are presented in 

10 terms of algorithms and symbolic representations of operations on data bits 
within a computer memory. These algorithmic descriptions and 
representations are the means used by those skilled in the data processing arts 
to most effectively convey the substance of their work to others skilled in the 
art. An algorithm is here, and generally, conceived to be a self-consistent 

15 sequence of steps leading to a desired result. The steps are those requiring 

physical manipulations of physical quantities. Usually, though not necessarily, 
these quantities take the form of electrical or magnetic signals capable of being 
stored, transferred, combined, compared, and otherwise manipulated. It has 
proven convenient at times, principally for reasons of common usage, to refer 

20 to these signals as bits, values, elements, symbols, characters, terms, numbers, 
or the like. 

It should be borne in mind, however, that all of these and similar terms 
are to be associated with the appropriate physical quantities and are merely 
convenient labels applied to these quantities. Unless specifically stated 
25 otherwise as apparent from the following discussion, it is appreciated that 
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throughout the description, discussions utilizing terms such as "processing" or 
"computing" or "calculating" or "determining" or "displaying" or the like, refer 
to the action and processes of a computer system, or similar electronic 
computing device, that manipulates and transforms data represented as 
5 physical (electronic) quantities within the computer system's registers and 

memories into other data similarly represented as physical quantities within the 
computer system memories or registers or other such information storage, 
transmission or display devices. 

The present invention also relates to apparatus for performing the 

10 operations herein. This apparatus may be specially constructed for the required 
purposes, or it may comprise a general-purpose computer selectively activated 
or reconfigured by a computer program stored in the computer. Such a 
computer program may be stored in a computer readable storage medium, 
such as, but is not limited to, any type of disk including floppy disks, optical 

15 disks, CD-ROMs, and magnetic-optical disks, read-only memories (ROMs), 
random access memories (RAMs), EPROMs, EEPROMs, magnetic or optical 
cards, or any type of media suitable for storing electronic instructions, and each 
coupled to a computer system bus. 

The algorithms and displays presented herein are not inherently related 

20 to any particular computer or other apparatus. Various general-purpose 

systems may be used with programs in accordance with the teachings herein, or 
it may prove convenient to construct more specialized apparatus to perform the 
required method steps. The required structure for a variety of these systems 
will appear from the description below. In addition, the present invention is 

25 not described with reference to any particular programming language. It will 
be appreciated that a variety of programming languages may be used to 
implement the teachings of the invention as described herein. 
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Overview 

A remote controller system that provides a user capabilities to retrieve 
multimedia objects is disclosed. The multimedia objects may be formed by 
using a combination of different formats such as, for example. Moving Picture 

5 Expert Group (MPEG) video's, MPEG audio Layer-3 (MPS), music compact 
discs (CD), etc. The multimedia objects are also referred to as media objects. In 
one embodiment, the system is a hand held device such as, for example, the 
Personal Digital Assistant (PDA) from Palm Inc. in Santa Clara, Ca., that allows 
the user to remotely select a media object from a network access appliance 

10 (NAA). The network access appliance is connected to a network allowing it to 
retrieve media objects stored in one or more servers. The servers may 
communicate with the NAA through the same network such as, for example, 
the Internet. In one embodiment, the NAA has a local cache that stores the 
media objects. For example, the NAA cache may store media objects that the 

15 user has identified as the user's favorites. This allows the NAA to quickly 
respond to the user's request when the user selects a favorite media object. 
When the NAA cache does not have the requested selection, the NAA retrieves 
the requested selection from a specified server. 

Figure 1 illustrates an exemplary network embodiment with the remote 

20 controller, the network access appliance and the server. When the user points 
the remote controller 130 at an NAA 120, 125, 128 the remote controller 130 
transmits sufficient information for the NAA 120, 125, 128 to identify the 
desired request from the user. In one embodiment, the communication 
between the remote controller 130 and the NAA 120, 125, 128 uses the Infrared 

25 Data Association (IrDA) Data Protocols specification with the Infrared Object 
Exchange Protocol (Ir OBEX) Data Protocol as published by the Infrared Data 
Association (IrDA). Using this protocol, the remote controller 130 sends a 
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resource identifier such as, for example, a uniform resource locator (URL), to 
the NAA 120, 125, 128 to request the NAA 120, 125, 128 to play the media object 
associated with the resource identifier. The NAA 120, 125, 128 then retrieves 
the media object from the network (e.g., Internet) or from a NAA local cache, 
5 sends an acknowledgement to the remote controller 130, and begins 
playing/ displaying the media object. 

In one embodiment, the remote controller 130 sends multiple URLs in 
one request and the NAA 120, 125, 128 plays each in order. Optionally, the 
remote controller 130 may send (e.g., upload) the actual media object to be 

10 played. In one embodiment, the remote controller 130 may send a request to 
the NAA 120, 125, 128 for the resource identifier of the currently playing object. 
The NAA 120, 125, 128 responds by sending the resource identifier and 
optionally a reduced graphical representation (e.g., thumbnail image) of the 
object. Optionally, the NAA 120, 125, 128 may also send data comprising the 

15 object. 

Depending on the type of NAA 120, 125, 128 and the request, the NAA 
120, 125, 128 may play the media object associated with the request when the 
media object is an audio clip or a video clip, or the NAA 120, 125, 128 may 
display the media object when the media object is a document. There may be 

20 different NAA types to handle the different formats of the media objects. 

In one embodiment, a media object is displayed as a thumbnail on the 
remote controller 130. The remote controller 130 may also store the resource 
identifier (e.g., URL) corresponding to each of the thumbnail images. For 
example, to play a desired media object the user selects the corresponding 

25 thumbnail image and presses a play button. The media object may need to be 
compatible with the NAA 120, 125, 128 in order for it to be played by the NAA 
120, 125, 128. In one embodiment, an error message may be sent by the NAA 
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120, 125, 128 to the remote controller 130 indicating a format incompatibility. 
For example, when the user selects a media object corresponding to a 
multimedia document while pointing the remote controller 130 to the car radio 
NAA 125, an incompatible format error message is received by the remote 

5 controller 130. The NAA 120, 125, 128 can be any device capable of receiving 
the requests from the remote controller 130. 

In one embodiment, each of the media objects stored or played by the 
NAA is associated with a resource identifier. For example, the stereo NAA 120 
of Figure 1 may be playing a song that it does not have in its local cache. Since 

10 the NAA 120 knows about the resource identifier associated with that song, the 
NAA 120 can retrieve the song from network server 105. In one embodiment 
the network server 105 stores multiple media objects. 

In another embodiment, a personalized server in the network is available 
to provide the users of the remote controller 130 a way to maintain the 

15 information stored on the remote controller 130. Each user of the remote 
controller 130 has a personalized server 110 such as, for example, "My PDA 
Server'' to keep track of all the services associated with the remote controller 
130. In one embodiment, ''My PDA Server" is implemented as a server 
directory corresponding to each remote controller 130, and all the different 

20 personalized servers are located in a server system. The user may add new 
resource identifiers to the personalized server 110, and the user may remove 
resource identifiers from the personalized server 110. In other words, the 
personalized server 110 keeps a copy of each resource identifier stored in the 
remote controller 130. Furthermore, the personalized server may also keep a 

25 copy of the media object corresponding to each of the resource identifiers. 

When the remote controller 130 is synchronized with the personalized 
server 110, all the thumbnail images and the corresponding resource identifiers 
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are loaded onto the remote controller 130. The process of synchronization may 
be done through a docking cradle connected to the Internet. In one 
embodiment, the NAA 120, 125, 128 periodically accesses the personalized 
server 110 and update the local cache in the NAA with the appropriate media 
5 objects based on the resource identifiers stored in the personalized server 110. 
The media objects associated with these resource identifiers may be located at 
different sites in the network. 

Figure 2 is a diagram illustrating one exemplary embodiment of 
accessing media objects on multiple NAA devices. Device 130A is a remote 

10 controller. In this diagram, the device 130A is the same remote controller as the 
device 130B, just at a different time and location, and is referred to as the 
remote controller 130. Initially, a copy of the media object 131 is stored in the 
NAA 120. This copy may be obtained by any number of methods such as, for 
example, downloaded from a server, copied over from the remote controller 

15 130, read off a physical media format such as, for example, a compact disc (CD), 
wireless transfer from a digital camera, or other means. Automatically or in 
response to a request from the remote controller 130, the NAA 120 makes a 
connection with the personalized server 110 and transfers a copy of the media 
object 131 to the personalized server 110. In one embodiment, the NAA 120 

20 may encrypt the media object 131 using a key obtained from the remote 

controller 130. In one embodiment, the remote controller 130 also provides a 
unique identifier for the media object 131 to the NAA 120. The NAA 120 then 
transfers this unique identifier a long with the media object 131 to the 
personalized server 110. In an alternative embodiment, the NAA 120 or the 

25 personalized server 110 assigns a unique identifier to the media object 131 and 
transmits this unique identifier back to the remote controller 130. 
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At some later time, the remote controller 130 may request the NAA 120 
to play the media object 131 by transmitting the unique identifier associated 
with the media object 131 to the NAA 128 as described above. The NAA 128 
may not be able to communicate directly with the NAA 120 to request the 

5 media object 131 due to the firewall 108 or other reasons. However, the NAA 
128 can communicate with the personalized server 110 and request the media 
object 131 using the unique identifier provided by remote controller 130. The 
personalized server 110 then sends the media object 131 to the NAA 128. The 
NAA 128 may optionally decrypt the media object 131 using a key provided by 

10 the remote controller 130. 

Figure 3 is a flow diagram illustrating an exemplary request to play a 
media object. The flow diagram starts at block 132. When the user activates a 
play function, such as, for example, pressing a play button on a remote 
controller, the remote controller sends one or more resource identifiers 

15 corresponding to the media object(s) to the specified NAA using a 

communication channel such as, for example, IrOBEX. At block 135, the NAA 
receives the resource identifier(s) and uses it as an identifier to locate the 
requested media object. For each resource identifier, the NAA first determines 
to see if the resource identifier for the requested media object may have already 

20 been stored in the cache within the NAA, as shown in block 140. When the 

requested media object is found in the cache, the NAA proceeds by sending an 
acknowledgement to the remote controller, as shown in block 155. 
Alternatively, when the requested media object is not foimd in the cache, the 
NAA attempts to locate the requested media object, using the resource 

25 identifier, in a network server (e.g., network server 105 of Figure 1), as shown 
in block 145. When the requested media object is not found in the cache or in 
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the network server, an error message is sent to the remote controller, as shown 
in block 150. 

At block 145, when the requested media object is found in the network 
server, an authentication may be performed to determine if the user is 

5 authorized to retrieve the media object from the network server, as shown in 
block 148. In one embodiment, the user is authenticated when the user has 
subscribed to the media object by paying an access fee. Authentication 
information, such as, for example, a user name and password may be sent to 
the NAA in conjunction with the requested media object. When the user is not 

10 authenticated, an authentication failure message is sent to the remote controller, 
as shown in block 150. The remote controller may re-send the same resource 
identifier request with the appropriate authentication such as, for example, the 
user name and password. When the user is authenticated, the requested media 
object is sent (e.g., downloaded) from the network server to the NAA, as shown 

15 in block 149. An acknowledgment may be sent by the NAA to the remote 
controller, as shown in block 155. The media object corresponding to the 
resource identifier is then played by the NAA, as shown in block 160. For 
example, when the NAA is a stereo receiver and the remote controller sends a 
resource identifier corresponding to a particular song, the NAA plays that song. 

20 When the NAA is a personal computer and the resource identifier sent by the 
remote controller corresponds to a particular document, the NAA displays that 
particular document. In block 165, a determination is made to see if there is any 
remaining resource identifier to be processed. When there are more resource 
identifiers, the flow returns to block 140 to process the next resource identifier. 

25 When all the resource identifiers have been processed, the flow stops at block 
170. 
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Figure 4 is a flow diagram illustrating one embodiment of an access 
authorization process performed by the network server. Figure 4 provides 
additional detailed information for the process performed in block 148 of 

5 Figure 3. As described in Figure 3, when the media object is not in the NAA, 
the NAA may send the resource identifier along with a user name and a 
password to the network server to request for the media object, as shown in 
block 405. At block 410, a determination is made to see if the requested media 
object does not exist in the network server and thus can not be received by the 

10 NAA. When that is the case, an error message is sent to the remote controller, 
as shown in block 415. However, when the requested media object exists in the 
network server, a copy of the requested media object is downloaded and stored 
in the NAA, as shown in block 422. The media object sent from the network 
server may or may not decrypted. If the media object is not encrypted, as 

15 determined by block 424, the flow moves to block 420 which ends at block 432, 
and the process is completed. 

When the media object sent by the network server is encrypted, the NAA 
requests for a private key from the remote controller, as shown in block 426. If 
the remote controller has the private key (e.g., the user of the remote controller 

20 is authorized), then it is used to decrypt the media object, as shown in blocks 
428 and 430. The process is completed at block 432. 

However, when the remote controller does not have the key, the process 
flows from block 428 to block 434. At block 434, the NAA sends a request for a 
decryption key to the network server. At block 438, the network server 

25 responds to the NAA with a public key of the network server and request the 
NAA to pay to access the requested media object. At block 442, the NAA 
forwards the network server public key and the payment request to the remote 
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controller. At block 446, the remote controller requests the user to accept the 
payment requirement from the network server. At block 448, a determination 
is made to see if the user accepts the payment requirement. If the user rejects 
the payment requirement, an error message may be generated by the NAA to 

5 indicate that the request to play the media object can not be satisfied, as shown 
in block 452. The process then flows to block 420 and is completed at block 432. 

If the user accepts the payment requirement, the process flows from 
block 448 to block 456. At block 456, the remote controller signs a payment 
information with a private key. The remote controller then encrypts the 

10 payment information with the network server public key. In one embodiment, 
the encrypted payment information is then sent to the network server via the 
NAA along with the original request for the media object. In another 
embodiment, the network server remembers the requested media object from 
the previous request, and the remote controller may only need to send the 

15 encrypted payment information to the network server. 

At block 460, the network server creates an encryption key for the 
requested media object. The network server also encrypts the encryption key 
for the requested media object with the private key of the remote controller. At 
block 464, the encrypted encryption key for the requested media object is sent 

20 by the network server to the remote controller via the NAA. At block 468, the 
remote controller decrypts the encryption key for the requested media object 
using the remote controller private key. The encryption key for the requested 
media object is then sent to the NAA. The process flows back to block 428 
where the NAA receives the encryption key for the requested media object 

25 from the remote controller. At block 430, the NAA uses the encryption key for 
the requested media object to decr3^ts the media object. The process is 
completed at block 432. 
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Figure 5 is an exemplary flow diagram for processing a request by a 
remote controller to acquire an object being played. The flow diagram starts at 
block 500. At block 505^ the remote controller sends a request to the NAA to 
5 acquire a media object being played on the NAA. For example, the user may 
initiate this request by pressing an ''add'' button on the remote controller. The 
media object may be played in response to a play command from another 
remote controller. At block 510, the NAA receives the request from the remote 
controller. At block 515, a determination is made to see if a media object is 

10 actually being played by the NAA. For example, the request from the remote 
controller may have been sent accidentally when the NAA is not playing any 
media object. Those skilled in the art would recognize that playing the media 
object may include, for example, displaying a document, playing an audio clip, 
playing a video clip, etc. When there is no media object being played by the 

15 NAA, an error message is sent to the remote controller, as shown in block 520, 
and the process ends at block 540. 

In one embodiment, when there is a media object being played by the 
NAA, the network server or the NAA creates a unique resource identifier for 
the media object for that particular remote controller. The NAA then sends this 

20 unique resource identifier to the remote controller, as shown in block 525. The 
unique resoiirce identifier may be used to access/encode authentication 
information in addition to identify the media object. 

The remote controller may need to be authenticated in order to be able to 
subsequently send a request to play the same media object. For example, there 

25 may be a cost associated with having access to the media object. At block 530, a 
determination is made to see if authentication is required. When there is no 
such requirement (e.g., the media object is free), the process ends at block 540. 
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However, when the authentication is required, the process flows to block 535 
where the authentication process is performed. In one embodiment, the process 
performed in block 535 is similar to the process performed in the flow diagram 
of Figure 4. In another embodiment, the authorization and payment may 
5 happen at a later time when user requests to play the object. In this case, the 
process performed in blocks 530 and 535 are bypassed and the block 525 flows 
directly to block 540. 

Figure 6 illustrates an exemplary embodiment of a remote controller. In 
this embodiment, there are three tracks 205, 210, 215 with each track having 

10 multiple numbers of thumbnail images 220. In one embodiment, a thumbnail 
image can be selected by using navigating arrows 235 on the remote controller 
200. When a desired thumbnail image is selected, and a play button 230 is 
pressed, the remote controller 200 sends a signal containing the resource 
identifier out to the NAA. The resource identifier corresponds to the selected 

15 thumbnail image. In one embodiment, the outgoing signal is sent through an 
output port designed for wireless conununication. 

The exemplary remote controller 200 has three tracks 205, 210 and 215 to 
allow the user to organize the media objects (thumbnail images). This provides 
the user an easy way to organize the media objects by the track. For example, 

20 the media objects displayed on the first track 205 may be a scrollable list with all 
the media objects that are known to the remote controller 200. The user can 
select one of the media objects 220 on any track using the navigation controls. 
In one embodiment, the third track 215 is used as a working track. For 
example, the user can create a play list containing the user favorite songs by 

25 selecting music media objects from the first track 205 and add them to the third 
track 215. When the user is satisfied with the play list, the user can move the 
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play list to the second track 210. In one embodiment, each thumbnail image on 
the second track 210 represents a play list created by the user. 

When the remote controller is synchronized with the personalized server 
110, the play list is updated in the personalized server 110 and the indicated 

5 media objects may eventually be accessed and cached by the appropriate NAA. 
In one embodiment, the synchronization is done periodically such as, for 
example, every 24 hours. This capability allows the user to easily find and 
select individual media objects or entire play lists on the remote controller and 
request an NAA to play them. When the NAA has cached the media objects (by 

10 examining stored play lists) the user will not have to wait for any downloads 
from the network server. Other functions available on the remote controller 
may include ''stop'', "pause", etc. to provide the user additional control 
capabilities. It would be apparent to one skilled in the art that there may be 
other ways of organizing the media objects to make them easily retrievable. For 

15 example, the media objects may be organized using two or four tracks, in a 
hierarchical list or just a list of the resource identifiers. 

In one embodiment, the remote controller 200 of Figure 6 has an "add" 
button 225 that allows the user to add the resource identifier of the song 
currently played to the remote controller 200. Upon receiving the "add" 

20 request signal from the remote controller 200, the NAA responds with the 
resource identifier information. Similarly, the resource identifier for a 
multimedia document is transmitted when the document is being displayed by 
a document NAA. In one embodiment, each thumbnail is stored along with 
additional information about the associated media object such as, for example, 

25 the date the media object was created or authentication and payment 

information. In another embodiment, the remote controller 200 provides the 
user a capability to generate text information about the media objects. For 
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example, the user may use the text input area 240 to add notes or comments 
about the selected media object. 

In yet another embodiment, the remote controller 200 may also include 
the ability for the user to record audio narratives associated with the selected 
5 media object. Additionally, the remote controller 200 may also include the 
ability to communicate with other remote controllers to exchange or share 
media objects. For example, the user of the remote controller may send a 
favorite play list to another user. 

Figure 7 illustrates an exemplary representation of a remote controller 

10 with a stereo network access appliance. In one embodiment, the remote 

controller 300 is a PDA. The PDA 300 has three tracks of thumbnail images 305. 
Each thumbnail image can be associated with a media object and a 
corresponding resource identifier. The communication channel 310 between 
the PDA 300 to the stereo NAA 320 is bi-directional. This allows the NAA 320 

15 to receive commands or requests from the PDA 300 and to transmit information 
to the PDA 300. For example, the PDA may transmit the resource identifier of 
the song that the user wants to add to the play list on the PDA 300. The stereo 
NAA 320 is connected to the Internet 325 to periodically update the content of 
its cache. 

20 Figure 8 illustrates an exemplary representation of a remote controller 

with a document network access appliance. The PDA 400 commimicates with 
the document NAA 410 through the communication channel 405. As discussed 
previously, the communication channel 405 may be a wireless channel. The 
NAA 410 may be any device capable of displaying a document such as, for 

25 example, a personal computer. The NAA 410 accesses the Internet to retrieve 
necessary information to update its cache storage. This updating process 
allows the NAA 410 to respond quickly to the user's request. In one 
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embodiment/ when cache storage is large enough, the user may have all of the 
requests satisfied locally without the NAA 410 having to access the Internet. It 
would be apparent to one skilled in the art that the NAA 410 can be any 
Internet connected device capable of playing the media object of a compatible 

5 format. For example, the NAA 410 can be a television that can be used to play a 
video related media object. 

From the above description and drawings, it will be understood by those 
of ordinary skill in the art that the particular embodiments shown and 
described are for purposes of illustration only and are not intended to limit the 

10 scope of the invention. Those of ordinary skill in the art will recognize that the 
invention may be embodied in other specific forms without departing from its 
spirit or essential characteristics. References to details of particular 
embodiments are not intended to limit the scope of the claims. 
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CLAIMS 

What is claimed is: 

1 1. A system comprising: 

2 a controller configured to select an identifier associated with a media object 

3 to send a request to play the media object, wherein the controller 

4 sends the request by transmitting the identifier stored in the 

5 controller; and 

6 an appliance configured to receive the request from the controller, to 

7 retrieve the media object from a first server via a network connection 

8 when the media object is not stored in the appliance, and to play the 

9 media object. 

1 2. The system of claim 1, wherein the identifier is transmitted using a wireless 

2 connection. 

1 3. The system of claim 1, wherein the controller and the first server are 

2 synchronized on a predetermined time period. 

1 4, The system of claim 3, wherein the first server stores the media objects 

2 corresponding to the identifiers stored in the controller. 

1 5. The system of claim 1, wherein the media object is retrieved from the first 

2 server using the identifier received from the controller. 
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1 6. The system of claim 1, further comprising a second server coupled to the 

2 network, the second server storing at least the media objects stored in the 

3 first server. 

1 7. The system of claim 6, wherein the appliance is further configured to 

2 retrieve the media object from the second server when the media object is 

3 not found in the first server. 

1 8. The system of claim 7, wherein the media object retrieved from the second 

2 server is in a decrypted form. 

1 9. The system of claim 7, wherein the media object retrieved from the second 

2 server is in an encrypted form. 

1 10. The system of claim 9, wherein a decryption key for the media object is 

2 stored in the controller. 

1 11 . The system of claim 10, wherein the decryption key is stored in the 

2 controller after the controllers sends a payment information to the second 

3 server. 

1 12. The system of claim 11, wherein the appliance receives the decryption key 

2 from the controller to decrypt the media object. 

1 13. The system of claim 1, wherein the identifier is selected by selecting a 

2 visual representation of the identifier. 
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1 14. The system of claim 13, wherein the visual representation comprises a 

2 thumbnail image representing the media object. 

1 15. The system of claim 14, wherein the controller organizes thumbnail images 

2 in groups. 

1 16. The system of claim 15, wherein the groups comprise: 

2 a first group including all thumbnail images stored in the controller, and 

3 a second group including selected thumbnail images from the first group. 

1 17. The system of claim 16, wherein the second group comprises: 

2 a first subgroup including one or more playlists, each of the playhsts 

3 comprising one or more thumbnail images; and 

4 a second subgroup including one or more thumbnail images in a playlist 

5 being created. 

1 18. The system of claim 17, wherein the controller sends one play list to the 

2 appliance to request the one play list be played by the appliance. 

1 19. The system of claim 17, wherein the controller comprises a display screen 

2 to display thumbnail images in the first group and in the second group. 

1 20. The system of claim 19, wherein the controller further comprises a 

2 microphone to record an audio annotation associated with one of the 
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3 thumbnail images, and a text input area to generate text to associate with 

4 the one thumbnail image. 

1 21. The system of claim 1, wherein the appliance is operable to play a media 

2 object not stored in the controller, and wherein the controller imports the 

3 identifier associated with the media object by sending a request to import 

4 the identifier not stored in the controller. 

1 22. The system of claim 21, wherein in response to the request to import the 

2 identifier not stored in the controller, the appliance sends the identifier and 

3 a reduced visual representation of the corresponding media object. 

1 23. The system of claim 22, wherein the reduced visual representation is a 

2 thumbnail image of the corresponding media object. 

1 24. The system of claim 21, wherein the request to import the identifier not 

2 stored in the controller is sent with payment information. 

1 25. The system of claim 1, wherein the appliance stores the media object in a 

2 cache. 

1 26. The system of claim 1, wherein the appliance is one in a group comprising 

2 a personal computer, a stereo receiver, and a television. 

1 27. The system of claim 26, wherein the controller operates with multiple 

2 appliances. 
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1 28. The system of claim 1, wherein the media object is one in a group 

2 comprising a document, an audio clip and a video clip. 

1 29. A system comprising: 

2 first means for selecting an identifier associated with a media object to 

3 initiate a request to play the media object, the first means transmitting 

4 the identifier stored in the first means; 

5 second means for retrieving the media object using the identifier and 

6 playing the media object; and 

7 third means for storing the media object, wherein the second means 

8 retrieves the media object from the third means at certain times, via a 

9 network, when the media object is not stored in the second means. 

1 30. The system of claim 29, wherein the first means transmits the identifier 

2 using a wireless connection. 

1 31 . The system of claim 29, further comprising fourth means coupled to the 

2 network, the fourth means for providing the media object when the media 

3 object is not in the third means. 

1 32. The system of claim 31, further comprising means for perfornning access 

2 authorization when the media object is retrieved from the fourth means. 
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1 33. The system of claim 32, wherein the means for performing access 

2 authorization comprises means for encrypting the media object and means 

3 for decrypting the media object. 

1 34. The system of claim 29, wherein the third means for storing the media 

2 objects comprises means for synchronizing with the first means to enable 

3 the first means to have the identifiers associated with the media objects 

4 stored in the third means. 

1 35. The system of claim 29, wherein the first means is operable with one or 

2 more second means. 

1 36. The system of claim 29, wherein the first means comprises means for 

2 organizing the identifiers using thumbnail image representations of the 

3 media objects associated with the identifiers. 

1 37. An apparatus comprising: 

2 means for transmitting a first identifier associated with a first media object 

3 to request the first media object be played; 

4 means for acquiring a second identifier associated with a second media 

5 object while the second media object is being played; and 

6 means for organizing the first and the second identifiers. 

1 38. The apparatus of claim 37, further comprising means for storing the first 

2 and the second identifiers. 
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1 39. The apparatus of claim 37, wherein the means for acquiring the second 

2 identifier comprises means for transmitting a payment information. 

1 40. The apparatus of claim 39, wherein the means for acquiring the second 

2 identifier further comprises means for generating a decryption key when 

3 the second media object is in an encrypted form. 

1 41 . The apparatus of claim 37, wherein the means for organizing the first and 

2 the second identifiers comprises: 

3 means for displaying the first media object and the second media object as 

4 thumbnail images, and 

5 means for organizing the thumbnail images in groups. 

1 42. The apparatus of claim 41, wherein the groups comprise: 

2 a first group comprising the thumbnail images, and 

3 a second group comprising subgroups of selected thumbnail images from 

4 the first group. 

1 43. The apparatus of claim 42, wherein the second group comprises: 

2 a first subgroup including one or more playlists, each of the playlists 

3 comprising one or more thumbnail images; and 

4 a second subgroup including one or more thumbnail images in a playlist 

5 being created. 
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ABSTRACT OF THE DISCLOSURE 

A method and system that organize and retrieve multimedia objects are 
described. In one embodiment, a controller is configured to select an identifier 
associated with a media object to send a request to play the media object. The 
controller sends the request by transmitting the identifier stored in the 
controller. An appliance is configured to receive the request from the controller. 
The appliance retrieves the media object from a first server via a network 
connection when the media object is not stored in the appliance. The appliance 
plays the media object. 
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APPENDIX B 



Title 37, Code of Federal Regulations, Section 1 .56 
Duty to Disclose Information Material to Patentability 

(a) A patent by its yery nature is affected with a public interest. The public interest is best served, 
and the most effective patent examination occurs when, at the time an application is being examined, the 
Office is aware of and evaluates the teachings of all Information material to patentability. Each Individual 
associated with the filing and prosecution of a patent application has a duty of candor and good faith in 
dealing with the Office, which Includes a duty to disclose to the Office all information known to that individual 
to be material to patentability as defined in this section. The duty to disclosure information exists with respect 
to each pending claim until the claim Is cancelled or withdrawn from consideration, or the application becomes 
abandoned. Information material to the patentability of a claim that is cancelled or withdrawn from 
consideration need not be submitted if the information is not material to the patentability of any claim 
remaining under consideration in the application. There is no duty to submit information which Is not material 
to the patentability of any existing claim. The duty to disclosure all Information known to be material to 
patentability is deemed to be satisfied If all Information known to be material to patentability of any claim 
issued in a patent was cited by the Office or submitted to the Office in the manner prescribed by §§1 .97(b)-(d) 
and 1 .98. However, no patent will be granted on an application in connection with which fraud on the Office 
was practiced or attempted or the duty of disclosure was violated through bad faith or intentional misconduct. 
The Office encourages applicants to carefully examine: 

(1) Prior art cited in search reports of a foreign patent office in a counterpart application, and 

(2) The closest information over which individuals associated with the filing or prosecution of a 
patent application believe any pending claim patentably defines, to make sure that any material information 
contained therein is disclosed to the Office. 

(b) Under this section, information Is material to patentability when it is not cumulative to 
information already of record or being made or record in the application, and 

(1) It establishes, by itself or in combination with other information, a prima facie case of 
unpatentability of a claim; or 

(2) it refutes, or is inconsistent with, a position the applicant takes In: 

(i) Opposing an argument of unpatentability relied on by the Office, or 

(ii) Asserting an argument of patentability. 

A prima facie case of unpatentability is established when the information compels a conclusion that a claim Is 
unpatentable under the preponderance of evidence, burden-of-proof standard, giving each term in the claim 
its broadest reasonable construction consistent with the specification, and before any consideration is given to 
evidence which may be submitted in an attempt to establish a contrary conclusion of patentability. 

(c) Individuals associated with the filing or prosecution of a patent application within the 
meaning of this section are: 

(1) Each inventor named in the application; 

(2) Each attorney or agent who prepares or prosecutes the application; and 

(3) Every other person who is substantively involved in the preparation or prosecution of the 
application and who is associated with the inventor, with the assignee or with anyone to whom there is an 
obligation to assign the application. 

(d) Individuals other than the attorney, agent or inventor may comply with this section by 
disclosing information to the attorney, agent, or Inventor. 
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